Consensus Formation in a Two-Time-Scale Markovian System

This work analyzes distributed linear averaging within a connected network of sensors that each track the stationary distribution of an ergodic Markov chain with a slowly switching regime. Our approach is based on a two-time-scale stochastic approximation. A hyperparameter modeled as a Markov chain on a slower time-scale modulates the regime of each observed Markov chain. The average of all currently observed stationary distributions constitutes the average-consensus estimate to be reached by all sensors. Assuming the Markov chains do not share a common stationary distribution conditioned on their regime, then under the proposed linear averaging algorithm, the exchange graph conditions required for the sequence of sensor state values to converge weakly to the average-consensus are obtained. Estimation of a weighted average of all observed stationary distributions, not only the current ones, is proved feasible over a long-run time horizon, provided an additional communication condition holds. The sensor state values are also shown to converge weakly to solutions of a differential inclusion when the communication exchange graphs or observed Markov chains belong to a family of possible values, thus leading to a set-valued consensus formation. The rate of convergence of the consensus algorithm is studied by considering the scaled tracking errors when oriented about their steady-state for each regime of the hyperparameter. In addition, a Brownian bridge limit is obtained for a centered and scaled sequence of empirical measures. An adaptation rate is proposed as the minimum exponential rate of the sensor trajectories to the averageconsensus estimate. Various optimization problems related to this adaptation rate are posed, as well as an approximate ratio that relates between any two sets of exchange graphs the adaptation rate, sensor scaled error, and absolute sum total averaging weights. Simulations illustrate our results and observation model.